{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "(application)=\n",
    "# Application runtime\n",
    "\n",
    "You can use the {py:meth}`~mlrun.runtimes.ApplicationRuntime` to provide an image that runs on top of your deployed model. \n",
    "\n",
    "The application runtime deploys a container image (for example, a web application) that is exposed on a specific port, and a command to run the HTTP server. The runtime is based on top of Nuclio, and adds the application as a side-car to a Nuclio function pod while the actual function is a reverse proxy to that application. \n",
    "\n",
    "You can set an existing image to run in the application, or let the application runtime build the side-car image for you, by specifying the source code, and pulling at build-time. \n",
    "\n",
    "> Note: The default base image is python:3.11 (or 3.9 based on the client python version) when not specifying an image for the application.\n",
    "    \n",
    "An API Gateway, by default, is in front of the application and can provide different authentication methods, or none.\n",
    "\n",
    "Typical use cases are:\n",
    "- Deploy a [Vizro](https://github.com/mckinsey/vizro) dashboard that communicates with an external source (for example, a serving model) to display graphs, data, and inference.\n",
    "- Deploy a model and a UI &mdash; the model serving is the backend and the UI is the side car.\n",
    "- Deploy a fastapi web-server with an MLRun model. In this case, the Nuclio function is a reverse proxy and the user web-app is the side car.\n",
    "\n",
    "**In this section**\n",
    "- [Usage examples](#usage-examples)\n",
    "- [Authentication modes](#authentication-modes)\n",
    "- [Expose multiple ports](#expose-multiple-ports)\n",
    "- [Application runtime in a dark environment](#application-runtime-in-a-dark-environment)\n",
    "- [Application and serving function integration](#application-and-serving-function-integration)\n",
    "\n",
    "## Usage examples\n",
    "\n",
    "Deploy a Vizro dashboard from a pre-built image:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Create an application runtime (with pre-built image)\n",
    "application = project.set_function(\n",
    "    name=\"my-vizro-dashboard\", kind=\"application\", image=\"repo/my-vizro-image:latest\"\n",
    ")\n",
    "# Set the port that the side-car listens on\n",
    "application.set_internal_application_port(port=8050)\n",
    "\n",
    "# Deploy\n",
    "application.deploy()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Deploy a Vizro dashboard from a source archive or git:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Specify the source to be loaded at build-time or run-time\n",
    "application = project.set_function(\n",
    "    name=\"my-vizro-dashboard\", kind=\"application\", requirements=[\"vizro\"]\n",
    ")\n",
    "application.set_internal_application_port(8050)\n",
    "application.spec.command = \"gunicorn\"\n",
    "application.spec.args = [\n",
    "    \"<my-app>:<my-server>\",\n",
    "    \"--bind\",\n",
    "    \"0.0.0.0:8050\",\n",
    "]\n",
    "\n",
    "# Provide code artifacts\n",
    "application.with_source_archive(\n",
    "    \"git://github.com/org/repo#my-branch\", pull_at_runtime=False\n",
    ")\n",
    "\n",
    "# Build the application image via MLRun and deploy the Nuclio function\n",
    "# Optionally add mlrun\n",
    "application.deploy(with_mlrun=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "Reusing an already built reverse proxy image is done when:\n",
    "- Redeploying a function that built the reverse proxy once and has `application.status.container_image` enriched.\n",
    "- It was already built manually with `mlrun.runtimes.ApplicationRuntime.deploy_reverse_proxy_image()`.\n",
    "\n",
    "Using one of the above options can help minimize redundant builds and streamline the development cycle."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Authentication modes\n",
    "\n",
    "An application runtime can be accessed through an API Gateway that supports various authentication methods. \n",
    "\n",
    "The default authentication mode is `none` for open source and `access-key` for the Iguazio platform.\n",
    "\n",
    "The different authentication modes can be configured as follows (see {py:meth}`~mlrun.runtimes.nuclio.api_gateway.APIGateway` for further information):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from mlrun.common.schemas.api_gateway import APIGatewayAuthenticationMode\n",
    "\n",
    "# Unless disabled, the default API gateway is created when the application is deployed\n",
    "application.deploy(create_default_api_gateway=False)\n",
    "\n",
    "# Create API gateway without authentication\n",
    "application.create_api_gateway(\n",
    "    authentication_mode=APIGatewayAuthenticationMode.none,\n",
    ")\n",
    "\n",
    "# Basic authentication mode.\n",
    "# This means that the application can be invoked only using the provided credentials\n",
    "application.create_api_gateway(\n",
    "    authentication_mode=APIGatewayAuthenticationMode.basic,\n",
    "    authentication_creds=(\"my-username\", \"my-password\"),\n",
    ")\n",
    "\n",
    "# Access-key authentication mode. the application can be invoked only with a valid session\n",
    "application.create_api_gateway(\n",
    "    authentication_mode=APIGatewayAuthenticationMode.access_key,\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### API gateway configurations\n",
    "\n",
    "If you want to deploy an application with the default API Gateway, simply run `app.deploy()`. \n",
    "If you don’t need the default API Gateway, or if you prefer to create your own custom one, set `create_default_api_gateway=False` when calling `deploy()` and then manually create a [custom API Gateway](../concepts/nuclio-real-time-functions.ipynb).\n",
    "If you specify a custom port, you must set `direct_port_access=True`; otherwise, the value is ignored and the internal application port is used instead.\n",
    "\n",
    "There are additional parameters that can be configured when creating an API gateway:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "application.create_api_gateway(\n",
    "    # The name of the API gateway\n",
    "    name=\"my-api-gateway\",\n",
    "    # Optional path of the API gateway, default value is \"/\". The given path should be supported by the deployed application\n",
    "    path=\"/\",\n",
    "    # Set to True to allow direct port access to the application sidecar\n",
    "    direct_port_access=False,\n",
    "    # Set to True to force SSL redirect, False to disable. Defaults to mlrun.mlconf.force_api_gateway_ssl_redirect()\n",
    "    ssl_redirect=True,\n",
    "    # Set the API gateway as the default for the application (`status.api_gateway`)\n",
    "    set_as_default=False,\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Expose multiple ports\n",
    "\n",
    "```{admonition} Note\n",
    "Requires Nuclio 1.14.14 and higher.\n",
    "```\n",
    "\n",
    "By default, MLRun creates an API gateway for each application that forwards the events to the internal port, in this example 8010.\n",
    "Some applications use different ports, for example, for the application API and the dashboard. \n",
    "This example uses the default port 8010 for the application and adds port 8020 for other uses.\n",
    "\n",
    "```\n",
    "# Enable direct_port_access\n",
    "direct_port_access=True\n",
    "# Set multiple ports\n",
    "app.with_sidecar(ports=[8010,8020])\n",
    "# Set the internal (main) port\n",
    "app.set_internal_application_port(8010)\n",
    "# Add API gateway for second port\n",
    "url_direct = app.create_api_gateway(port=8020,name=\"port-8020\",direct_port_access=True,authentication_mode=mlrun.common.schemas.api_gateway.APIGatewayAuthenticationMode.none)\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Application runtime in a dark environment\n",
    "\n",
    "To use application runtime in a dark (air-gapped) environment, you need to build the reverse proxy image and push it to a private registry, following the steps below:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 1. Create the reverse proxy image in a non air-gapped system\n",
    "import mlrun.runtimes\n",
    "\n",
    "mlrun.runtimes.ApplicationRuntime.deploy_reverse_proxy_image()\n",
    "\n",
    "# 2. The created image name is saved on the ApplicationRuntime class:\n",
    "mlrun.runtimes.ApplicationRuntime.reverse_proxy_image\n",
    "\n",
    "# 3. Push the created image to the system’s docker registry\n",
    "\n",
    "# 4. On the air-gapped environment, set the image on the class property:\n",
    "mlrun.runtimes.ApplicationRuntime.reverse_proxy_image = (\n",
    "    \"registry/reverse-proxy-image:<tag>\"\n",
    ")\n",
    "\n",
    "# 5. When creating application functions, this image will be used as the reverse proxy image, and it won’t be built again."
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "## Application and serving function integration\n",
    "This example demonstrates deploying a serving function and using it with the application.\n",
    "</br></br>\n",
    "<u>Serving creation</u>:</br>\n",
    "First, create the model file and save it as a .py file."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "%%writefile add_ten_model.py\n",
    "\n",
    "from mlrun.serving import V2ModelServer\n",
    "\n",
    "class AddTenModel(V2ModelServer):\n",
    "    def load(self):\n",
    "        # No actual model to load, just a demo\n",
    "        pass\n",
    "\n",
    "    def predict(self, request):\n",
    "        input = request['inputs'][0]\n",
    "        result = input + 10\n",
    "        return {\"outputs\": [result]}"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "Now, deploy this model as a serving function:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "import mlrun\n",
    "\n",
    "project_name = \"app-demo-flask\"\n",
    "model_name = \"add-ten-model\"\n",
    "model_path = \"add_ten_model.py\"\n",
    "\n",
    "project = mlrun.get_or_create_project(project_name)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# Create the serving function\n",
    "# Create the serving function\n",
    "function = project.set_function(\n",
    "    name=\"add-ten-serving\",\n",
    "    kind=\"serving\",\n",
    "    func=model_path,\n",
    ")\n",
    "project.save()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# Add the model to the function (even though there's no real model in this case)\n",
    "function.add_model(model_name, model_path=model_path, class_name=\"AddTenModel\")\n",
    "function.deploy()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# An example of invoke:\n",
    "function.invoke(f\"/v2/models/{model_name}/infer\", {\"inputs\": [20]})[\"outputs\"][\n",
    "    \"outputs\"\n",
    "]"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "<u>Application creation</u>:</br>\n",
    "First, create the flask server application.</br>\n",
    "The application includes several different endpoints - to check its functionality:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "%%writefile flask_app_example.py\n",
    "\n",
    "from flask import Flask\n",
    "import requests\n",
    "import mlrun\n",
    "\n",
    "app = Flask(__name__)\n",
    "\n",
    "\n",
    "@app.route(\"/internal\")\n",
    "def internal():\n",
    "    # Test access to the serving function with MLRun.\n",
    "    project_name = \"app-demo-flask\"\n",
    "    project = mlrun.get_or_create_project(project_name)\n",
    "    function = project.get_function(\"add-ten-serving\",ignore_cache=True)\n",
    "    response = function.invoke(\"/v2/models/add-ten-model/infer\", {\"inputs\": [20]})\n",
    "    output = response[\"outputs\"][\"outputs\"]\n",
    "    return {\"result\": output}\n",
    "\n",
    "\n",
    "@app.route(\"/external\")\n",
    "def external():\n",
    "    # Test access to the serving function without MLRun (externally).\n",
    "    project_name = \"app-demo-flask\"\n",
    "    project = mlrun.get_or_create_project(project_name)\n",
    "    function = project.get_function(\"add-ten-serving\",ignore_cache=True) \n",
    "    url = f\"https://{function.status.external_invocation_urls[0]}\"\n",
    "    response = requests.post(url, json={\"inputs\": [50]}).json()\n",
    "    output = response[\"outputs\"][\"outputs\"]\n",
    "    return {\"result\": output}"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "Archive the application code into a .tar.gz file:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "!tar -czvf archive.tar.gz flask_app_example.py"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "Now, deploy this flask application:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "import mlrun\n",
    "\n",
    "project = mlrun.get_or_create_project(\"app-demo-flask\")\n",
    "# Create a demo secret for testing\n",
    "project.set_secrets(secrets={\"secret-example\": \"project_secret_example\"})"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# Specify source to be loaded on build time\n",
    "# The image or the requirements should include mlrun, flask and gunicorn.\n",
    "application = project.set_function(\n",
    "    name=\"flask_app\",\n",
    "    kind=\"application\",\n",
    "    requirements_file=\"/your/path/requirements.txt\",  # if needed\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# Provide code artifacts\n",
    "application.with_source_archive(\n",
    "    \"v3io:///your/path/archive.tar.gz\", pull_at_runtime=False\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "application.set_internal_application_port(5000)\n",
    "application.spec.command = \"gunicorn\"\n",
    "application.spec.args = [\n",
    "    \"flask_app_example:app\",\n",
    "    \"--bind\",\n",
    "    \"127.0.0.1:5000\",\n",
    "    \"--log-level\",\n",
    "    \"debug\",\n",
    "]\n",
    "application.deploy(with_mlrun=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2024-08-15T10:24:12.830539Z",
     "start_time": "2024-08-15T10:24:12.825243Z"
    },
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# Test the deployment:\n",
    "application.invoke(\"/internal\").json()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "application.invoke(\"/external\").json()"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.13"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}
